The top stops along the way from Pondicherry to Madurai (with short detours) are Promenade Beach, Brihadeeswara Temple, and Paradise Beach Puducherry. Other popular stops include Sri Ranganatha Swamy Temple, Srirangam, Palani Murugan Pathaiyarhirai mandapam, and Arulmigu Manakula Vinayagar Devasthanam.
The top stops along the way from Pondicherry to Rameswaram (with short detours) are Sri Arulmigu Ramanathaswamy Temple, Brihadeeswara Temple, and Dhanushkodi Beach. Other popular stops include Dharasuram Airavatesvara Temple, Pichavaram Mangrove Forest, and Gangaikonda Cholapuram.
You can visit the city for as many days as you want but if you come be prepared with the itinerary. Madurai 2 days packages are a great idea because you would not want to waste your time anyhow and also miss some attraction point of the place.
